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A dexterous robot manipulator must be able to feel what it is doing. She mechanical 
hand of the future will toll a screw between its fingers and sense, by touch, which end is wniclv, 
This paper describes a step toward such a manipulator-" a iobot hngci that is used to iccogni/e 
objects by touch, ihe device neorporates a novel imaging tactile sensei- an aitihcial skin 


Sinai 


with hundreds of pressure sensors in a space the si/e of a finger tip. The sensor is mounted on a 
tendon-actuated mechanical finger, similar in size and range of motion to a human index finger. A 


prog 


ram controls the finger, using it to press and probe the object placed in front of it. Based on 


how the object feels, the program guesses its shape and orientation and their uses the linger to test 
and refine the hypothesis. The device is programmed to recognize commonly used fastening 
devices- nuts, bolts, flat washers, lock washers, dowel pins, cotter pins, and set screws. 

The paper is divided into three main sections. 1 he first is a description of the tactile 
sensor array- how it is constructed and what it can do. 'The second section describes the 
mechanical finger and how it is controlled. The final section of the main portion of the paper is a 
description of a program that uses the finger and sensor to recognize small objects. Finally, there 
is an appendix, describing some related work on tendon hands and arms. 




2, The Sensor 


The touch sensor is a monolithic array of 256 tactile sensors that fits (appiopiiately) on 
the tip of a finger. This is comparable to the resolution of the human forefinger, h.ach scnsoi has 


an area 


of less than one hundredth of a square centimeter and gives an independent anaiog 


indication of the force over its surface in the range of 1 to 100 grams, t he ar ay is scanned one 
column at a time to minimize the number of connecting wires. The sensor is rugged, flexible, and 


has a skin-like texture. 


2.! Method of Construction 




The touch array has two conductive components: a flexible printed circuit boaid <md a 
sheet of anisotropically conductive silicone rubber (ACS). The ACS has tire peculiar property of 
being electrically conductive along only one axis in the plane of tire sheet, lire punted encuit 
board is etched into fine parallel lines, so it too conducts in only one dimension. The two 
components arc placed into contact with the lines on tire printed circuit board perpendicular to 
the ACS axis of conduction. The contact points at each intersection of the perpendicular 

conductors form the pressure sensors. 

The device must also include a separator to pull tire conducting layers apart when 
pressure is released. The sensitivity and range of the sensor depend largely on the construction of 
this intervening layer. For a large pressure range the best separator I tested was the woven mesh of 
a nylon stocking. For high sensitivity, a separator may be deposited directly onto ACS by 
spraying it with a fine mist of non-conductive paint. The conductive rubber presses through the 
separator so that tire area of contact, and hence the contact resistance, varies with the applied 


pressure. 


model of tile 1 conk 


The prcssurc/resistnncc relationship is non-linear, as shown in Figure 5. 1 do not have a 
iU t mechanism that quantitatively explains the change in resistance with applied 


pressure, however Figure 2. illustrates a plausible qualitative model. Pressure on the elastomeric 
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ACS deforms the material around the separator, allowing it to contact the metal oeiow. taiga 


pressures re 


esults in more deformation and larger contact areas. 11 the i existence qf the contUQt is 


proportioi 


tal to the contact area, the contact resistance will be inversely jelated to the applied 


■essurc. For the object recognition application, the non-linear response of the sensor was not a 


pi 


significant drawback, 


The ACS itself is constructed of layers of silicone rubber impregnated with cithci 


graphite 01 


•silver, alternating with similar non-conduclive layers, Hach layer is approximately z50 


microns 


thick. The layers are oriented at right angles to the plane of the sheet. Jn its normal 


commercial application, ACS is sliced into strips and used to make contact between printed-circuit 
boards. The linear resistivity, in the conducting direction, is on the order of kilohms pci 
centimeter for graphite-impregnated ACS. This is inconveniently high for building large sensors, 
and 1 was able to lower it to approximately 100 ohms per centimeter by electroplating it with gold, 
it is possible to plate only over the conductive silicone, so that the cross resistance remains 
gntially infinite. Silver-impregnated silicone rubber has a substantially lower bulk resistance, 
able to obtain the material in the proper form. The minimum resolution of 


cssen 


but 1 was un 


C 


omrncrcially available ACS is about 50 lines per centimeter. 

Wires are connected to die edges of the printed circuit board by soldering. I he ACb is 


mounted in such a way that its edges fold around the printed circuit, where they arc presseu 
against contact fingers on the other side (see Figure 1). A compound sensor with high range and 
good sensitivity may be constructed by placing a high range (nylon mesh) sensor behind a 


sensitive one 


, In this case, 1 eliminated the flexible circuit board in the front layer, so that the 


center layer of ACS was shared between the two arrays. (I have constructed a sensor of this type, 
but it was not used in conjunction with the mechanical finger.) 
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Arm 


Attaching wires only at the edges of the array reduces the number of necessary 
connections. This is important given the limited space of a mechanical finger. (My 256 cell sensor 
used 32 wires, si/.c #42, stranded. The resulting cable is less than 3 mm. in diameter.) The array 
is scanned by applying a voltage to one column at a time and measuring the current flowing in 
each row. A potential problem with this method is the introduction of phantom tactile images. 
When multiple points are activated simultaneously, it may appear that untouched point arc also 
conducting. This is the analog version of the crosspoint problem in xy-sc;,nned keyboards: if 
three out of four switches on a rectangle are closed, the fourth appears to be also. This happens 
because the path through the other three connections is electrically in parallel with the ghost 
connection. In keyboards, it is usually avoided by putting a diode at each point of intersection. 
This could be done for touch array also, but it would add considerably to the complexity of the 
device and it might also introduce undesirable mechanical stiffness. With resistiv e contacts it is 
theoretically possible to compute the actual resistances from the measured resistances by solving 
N equations in N unknowns [4], but tire technique tends to amplify errors due to inaccuracy of 
measurement, noise, and resistance along the conductive axis. Other researchers [9,1,8] have 
avoided the problem by attaching a separate wire to each sense point. This is impractical for high 
resolution arrays and, again, it limits mechanical flexibility. 

Instead I used the scheme illustrated in Figure 4. It is similar to the voltage mirror 
approach suggested by Purbrick [7]. A fixed voltage is placed on die column of interest, while all 
other cohimns arc held at ground potential to ground out any alternate paths. The rows are all 
held to ground potential also, by injecting whatever current is necessary to cancel the current 
injected by the active column. The value of the resistance of a crosspoint is inversely proportional 
to the current that is necessary to pull the corresponding row to ground potential. By this method, 
extraneous columns arc at the same potential as the rows (ground), so no current will flow through 


the unmeasured crosspoints. The holding currents depend only on the column drive voltages and 





the resistances in question. The entire array is scanned by measuring one column at a time, as 
described above. 

The method described is valid only if the crosspoint resistances are high compared to the 
linear resistances of the row and column lines, otherwise it is not possible to hold an entire row or 
column at a fixed potential. This effect may be understood by referring to the electrical model of a 
single row illustrated in Figure 3. The applied voltage (V) and the linear resistance Rj arc known, 
but the unknown resistance (Rp ) can not be determined unless the potential at node N n is 
known. This potential may be computed, in time proportional to the number of nodes, by first 
measuring the unknown resistances near the edge. The resistance of all the the unknown contacts 
may be determined by computing the successive two-port parameters of the subnetworks toward 
the edge of the unknown node. 


0 
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Z 


_ R n< 7 n-1 + R L> 
n Z n-1 + R n + R L 


Where V is the applied voltage, I n and I mn the measured currents, Z n the input impedance and 

G is the voltage transfer ratio of the network to the left of R n . 

For large arrays it .may actually be necessary to compute the resistances as shown, but as 
Jong a the linear resistance is low, as compared with the contact resistance, ihi: is not necessary, if 


the measured values arc used directly, then the worst case error for a row of N elements is- 

R measured R n + NR L 

R_ 


R. 


actual 


This is easy to determine because the worst case occurs when all contacts, except for the one being 
measured, are open. Other contact closures will only lower the potential of node N, v increasing 
the accuracy of the measurement. Ihc error may also be reduced by a fa.ctoi of two by making 
contact at both ends of the row. 


[ constructed several sensory arrays with varying range and resolution. Figure 5. shows 
prcssurc/rcsistance curves for two representative devices. Device # 1 has a sprayed separator 
(approximately 10^ dots per square centimeter). The separator of device # L is nylon mesh (Lcggs, 
Extra Sheer). Hie ACS used in device #1 was plated with gold on the contact side. Ail devices 
showed good mechanical durability and, after an initial settling period, stable electrical 
characteristics. (The first, prototype, almost a year old, shows no noticeable change in contact 
resistance.) The highest resolution device (# 1) was a 16 by 16 array, one centimeter m aiea. i his 
is the sensor used with the linger. Sample images of the top of a screw, an electronic connector, a 


1/8 inch ring, and a cotter pin arc shown in Figure 6. 











6. Sample Tactile images trom the oeiiswr 

(actual size objects are shown below images) 
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3. The Mechanical Finger 


The finger, on which the touch sensor is mounted, has approximately the same shape, 
size and range of motion as the human forefinger. lake the human finger, it has three joints: a two 
degree of freedom pivot at the base and two single degree of freedom hinge joints. These joints 
arc controlled by four pairs of tendons, which arc driven from four electric motors mounted 
behind the base of the finger. Torque and position are measured only at the motors, so that joint 
torques and angles must be computed as described below. 'The finger and associated motors are 
mounted on a fixed base, with a small platform extending just below the linger, on which objects 
may he placed for testing. 

The tendons of the finger are arranged in opposing pairs- one bends the joint, the other 
straightens it. A system of pulleys keeps the total length of each pair constant. This allows both 
ends of a tendon pair to be driven from a single motor. The lever arm of the tendon pulling 
against the joint is kept constant over all angles by winding the tendon over a pulley fixed to the 
joint. 

The bodv of the finger is constructed of flat aluminum sections which are held together 


by press-fitted dowel pins. A!! non-fixed pulleys arc mounted on precision ball bearings. This is 
not really mechanically necessary, but the reduced friction makes the computation of joint torques 

i 

i 

more accurate. 1 

The tendon material is braided Kevlar, I/64th inch diameter, coated with polyurethane 
for abrasion resistance. Kevlar was chosen over steel because the minimum bending radius is 
smaller for a given diameter of cable. It is the minimum bending radius of the tendon material 
that limits the minimum size of ihe finger. If a more flexible tendon material was used, the design 
could easily be scaled to one half its current size, even using commercially available ball bearings. 
Smaller lingers would require special bearings, for example, die jewel bearings that arc commonly 
used in watches. 


! ! S V 


total length of each tendon pair must be kept constant so that (he pair may be 
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driven by a single motor. 'Phis is accomplished by winding the tendons around pulleys in opposite 
directions as they pass through a joint. When the joint bends, one tendon winds around the pulley 
;ts much as the other unwinds. This guarantees that the sum of the lengths of the two tendons 


re m 


ains constant. Each tendon has a stiff spring inserted along its length to set die common-mode 


tension 


The lengths of the two terminal tendon pairs are not actually invariant over changes in 


the horizontal direction of the proximal joint. This slight length change is compensated tot by 


stretching in the tendon springs. I ic position coupling would be insignificant except that it 


coin. 


ijjcates the computation of forces on the finger. A change in the total length of the tenuous 


causes the spring to stretch, and this in turn puts additional forces on the joint, ihe effect coulu 
be eliminated completely by adding another pulley stage, or by using constant force springs, 
instead of conventional springs, to set the common mode tension on the pair. A constant force 


spring 


does not change the apply force as it is stretched, so a length change would make no 


difference. 

The motors used to drive the tendon were ring-shaped moving-coil torque inotois. this 
type of motor was originally developed for gyroscope applications where precise control of torque 
is critical. The motors have a large number of poles and commutatoi points to reduce cogging 
effects. The particular motors used to drive the finger have a torque constant of 2.6 ounce inches 
per amp, plus or minus 2%. The internal winding resistance of these motors is about 10 ohms. An 
important property of these motors is their relatively large torque constant. This allows the 
tendons to be driven from a pulley directly attached to the shaft of the motor, without intervening 
gearing. Any such gearing would have introduced problems with back-lash, moment 
multiplication, etc. The primary disadvantage of the motors was their relatively high mining mass. 
Tim problem might have been reduced by the use of printed circuit motor, but at the cost of 

additional turning friction. 

The angle of the motors is sensed by a coaxial potentiometer. The pulley sizes of the 
motor are such that the full range of joint motion may be achieved with less than a full rotation 
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any of the motors. This allows the use of high-resolution single-turn potentiometers. The resistive 


■e ! 


lenient in these potentiometers is made of conductive plastic, rather than wound wire, for 


increased resolution. The motors have no external velocity sensors. Instead, velocity is computed 
from the motor’s induced voltage, as described later. 


3.1 Relating Motor Motion to Joint Motion 


The tendons in the mechanical finger tire arranged to give indcpcnc ent control of the 
torque and angle of each joint. Optimally the finger would be controlled by four motors, with the 
angle and torque of each joint corresponding directly to die angle and torque of one of the 
motors. In practice, this is difficult to realize because the tendon controlling one joint must pass 
through other joints on the way to its insertion. The mechanical designs which arc easiest to 
implement have arbitrary non-linear couplings between tire motors and the joints. I chose a 
compromise: Bach joint has associated with it a primary motor. The angle of the joint is a linear 
function of the primary motor angle and the angles of the primary motors of the intervening 
joints. (It is actually possible, at least in principle, to design a tendon finger with no joint 
interactions. What is required is a means of passing a tendon through a bending joint without 
changing its length. Just going through the joint’s axis of rotation will almost work, except that all 
tendons have a minimum bending radius. A cable inside an incompressible sheath, with no pulley, 
is another possibility. The most successful robot hands so far [5,6] have been of this second type.) 

The linearity constraint makes computation of the joint angles a matter of simple matrix 


multiplication: 


m=Mj 


Where m is the motor angle vector, j is the joint angle vector, and IV! is a lower triangular 
matrix. Assuming that the joints arc numbered starting from the base of the finger, the (i.j)th 
entry of the matrix is the ratio of the radii of pulley for the ith tendon in the jth joint and the ilh 
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inoLor pulley. 


Because the iih tendon must pass through ail joints less than i and none greater than i, 
the matrix must be lower triangular, with no zeros in- the bottom. Since all the diagonal elements 
arc non-zero, M is invertible. This is important because the inverse matrix is used by the 
controller to compute the motor positions for a given set of joint positions. For the pulleys 


actually used the matrix was: 


r.oo 

0.67 

0.32 

0.32 


0 

0.67 

0.48 

0.32 


0 

0 

0.48 

0.32 


0 

0 

0 

0.48 


A similar equation relates the joint and motor torques. In this case the ter 


ms of the 


matrix can depend on the angle of the joints, so there is a different matrix for each linger position. 
In this case the matrix is upper triangular: changing the torque in the primary motoi of a joint will 
affect the torques in all proximal joints. This is the opposite of what happens with angles, A 


change in a joint’s angle affects the joints which come after it, a change in a joint s torque affects 
the joints which come before it. 

The diagonal terms of the torque matrix are the reciprocals of the diagonal tcims in the 
position matrix, that is, they are the ratios of tire radii of the motor pulleys to the joint pulleys. 
The off-diagonal terms depend on the angles of the joints and the mechanical layout of the device. 
The finger was designed to keep these terms positive at all angles. This lias control advantages. 
Consider the typical control problem of holding a joint angle constant while a varying force is 
applied at the tip. To stabilize the finger, an increase in force in the terminal joint must be 
accompanied by an increase in force in the proximal joints. This is exactly the effect of having 


positive off-diagonal terms in the upper half of the torque matrix. 1 he magnitude of these te 
such that they provide useful mechanical fcedforwaid. 


terms is 


1 Actually, the effective radii of the pulleys must he used for this calculation 
the tendon. 


The radius of She pulley plus the radius ol 
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3.2 Dynamics 


In analyzing the dynamics of the mechanical finger it is realistic to consider only the 
mass of the motors. The mass of the finger itself is small by comparison. 'The motors may be 
controlled in one of two modes: position mode or torque mode. For the purpose of analyzing the 
motor driving circuitry, the motor may be modeled electrically as an ideal se vo motor in series 
with a resistor. This means that the torque on the motor is proportional to the current passing 
through it. and that the angular vcbcity of the motor is proportional to the voltage across the 


minals minus a correction factor (proportional to the current) for the series resistance of the 


motor. In this application the current is controlled and the voltage is measure 1 to determine the 
angular velocity. 

Fquivalent drive circuits are shown in Figure 8. In torque control mode, the current 
through the motor is sensed as a voltage across a resistor, and servoed to a level proportional to 
the desired torque. In position control mode, the velocity of the motor is proportional to the 
positional error. This is accomplished by supplying a current proportional to the difference 
between voltage across the motor and the positional error voltage. The voltage is proportional to 
velocity, and the current proportional to torque, which is proportional to angular acceleration. 
Therefore, the acceleration is proportional to the velocity error. For simplicity, the series 
resistance correction circuitry is not shown. 


3.3 The Human Finger 


For comparison, it is useful to consider Lite source of inspiration for this design-- the 
human finger. There are three bones in each finger. 'Hie joint at the base of the finger is a two 


degree of freedom ball and socket joint, with the socket on the finger bone. Twisting is 


1. 1 am pleased to say that 1 had an opportunity to personally verify these well know anatomical facts by dissecting a 
human cada\cr. 1 would like to thank l ulls Medical School for lending me a hand. 











constrained by the ellipsoid shape of the joint; the ball is wider than it is tall. Lateral motion is 
about 15 degrees in either direction, and is due partly to the motion of the metacarpal bones in the 
hand. The finger can Ilex almost 90 degrees toward the palm, and extend about 10 degrees away 
from it. The second and third joints each have only a single degree of freedom, constrained by a 
grooved ball and mating projection in the socket. The range of motion is about 120 degrees in the 
medial joint and 90 degrees in the terminal joint. Both joints may be hyperextended by a few 
degrees. The terminal bone is much smaller than the others and tapers to a poit t. 

Many of the muscles that control the fingers arc located in the forearm. They are 
connected to the finger bones by long flat tendons running inside sheaths. There arc also small 
muscles in the paint that originate at the mctacarpals and insert directly into the phalanges. 
Hxactly how these actuators work together to to move the finger is not well understood; it. has 
been the topic of at least two recent doctoral theses [10,3]. The problems arc that the actuators are 
not used in simple opposing pairs, the way they attach to the bones is difficult to model, and the 


Fig. 9. Model of the Human Finger 
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joints themselves are complicated. Figure 9 shows a simplified, mechanical model of the joints and 


actuators. l : or a more accurate mechanical mode! of the human finger sec [12] or 
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ififtr,. 


4. T he Program 


The tendon linger and the touch sensor are controlled by a tactile recognition program. 
The program uses the finger to press and probe the object placed in front of it. Based on how the 
object feels, the program guesses the shape and orientation of the object. The device is 
programmed to recognize commonly used fastening devices- nuts, bolts, fiat washers, lock 
washers, dowel pins, cotter pins, and set screws. The program is written in Lisp and runs on a 
specially augmented Lisp Machine [13], with independent micro-processors fa control low-level 
input/output functions. 


4.0.1 Why Touch is Easier than Vi, son 


In writing this simple tactile recognition system I have retraced, in both technique and 
spirit, the steps of early researchers in machine vision. There is good reason to believe that these 
simple techniques have a better chance of working in the tactile domain than they did in the 
visual. For one thing, there arc far fewer data to be analyzed than in a visual image. This means 
that even with a high resolution tactile array, complex processing may be performed in real time. 
Another factor is that collection is more readily controlled. Since placement and pressure of the 
fingertip are controlled by the program, analyzing a tactile image is like analyzing a visual image 
with controlled background, illumination, and point of view. 

There is also a third factor responsible for making tactile recognition the easier task: The 
properties that we actually measure are very close, in kind, to the properties that we wish to infer, 
in vision, it is only possible discover mechanical properties (shape, orientation, absolute position) 
by deducing them from optical properties (shading, projection, reflectivity). In touch, we measure 


mechanical properties directly. 





in \ 


I would like to contrast here two possible approaches to sensory recognition, in touch or 
■ision. The first is what 1 will call the classical approach, in the classical appioach we strut an 


image of some sort, extract features of tire image and then, from tire features, abstract some kind 
of a model of what is shown in the image. The classic appioach is bottom-up oi data driven.. I he 
second approach, which I shall call the active approach, is top-down or knowledge driven. When 
taking the active approach, we begin with a theory of what is in the image. Based on that theoiy 
we make measurements or perform experiments to test validity of the hypothesis, i he results of 
the theory arc then analyzed using the same analytic techniques as the classical approach, and 
based on the result of the analysis, the hypothesis may be modified or confirmed, if it is modified, 
we try experiments to test the new hypothesis, and so on, until we arrive at a conclusion that 
aerecs with the data. The two approaches are represented schematically in Figure 8. 


e 


The choice of top-down, vs. bottom-up was a source of extended controversy among 
arly researchers in machine vision. Ihe general consensus today seems to be that piocessing at 



Fig. 10. Two Approaches to Recognition 


1 

SENSE 

V 

ANALYZE 

ABSTRACT 

l 




CLASSICAL approach 


ACTIVE APPROACH 





25- 


tlic lowest levels is bottom-up and processing at higher levels in top-down. This is consistent with 
what I am calling the active approach. 

In the tactile domain, I believe that the active approach will yield the best results. For 


one tiling, the information in a single image is often insufficient for recognition of the object. 
Moving the finger to make different measurements is the best, way of collecting enough data, and 
in order to decide how to move the finger, the program must have some expectation of what 
object it is feeling. The program should operate at all times with a hypothesis of what it is feeling. 
It recognizes the object by actively probing it with the finger, modifying its internal 
"hallucination" of the object to conform with the measured reality. This is what 1 mean by active 
touch sensing. 


4.1 The Domain: Nuts ant! Bolts 


For the purpose of testing the sensor, the finger and the recognition techniques, I choose 


a restricted range of test objects which the program would be expected to recognize. Specifically, l 
choose a set of commonly used mechanical fastening devices such as screws and dow el pins. One 
advantage of these objects is that they are important for potential industrial applications of 
robotics. Recognition of fasteners and determination of their position and orientation when they 
arc grasped by a manipulator is an important industrial problem. It is also a problem that is 
unlikely to be solved by machine vision because the hand obscures the object and because forces 
cannot be seen. In this domain tactile and visual sensing would complement each other well: 
vision for locating objects and measuring their absolute position, touch for sensing local shape, 
orientation, and forces once they arc grasped. 

The particular objects chosen for study were machine screws, set screws, flat washers, 


lock washers, dowel pins and cotter pins. They were chosen because they have simple shapes that 


arc easy to represent and easy to distinguish. Restricting the range of possible objects to such a 


mall and easily distinguishable subset makes the recognition task less difficult, but it also 
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introduccs the possibility that he recognition methods used arc not really generally applicable. 
This may be the ease here, but 1 have been aware of the danger and have tried to implement the 
various algorithms is such a way that they may be extended to a larger range of objects. 

The actual objects used for testing were all small (not more than 1/2 inch in any 

dimension). In general, the smallest commonly used size in a given category was used for testing. 

1 

For example, the machine screw' was size #0-80 by j/ 8 inch and the cottci pin was 1/2 inch long 
by 1/16 inch diameter. Using such small objects allowed the entire image :o be read in one 
impression of the sensor. This avoided the problem of coordinating multiple scnsQi impiessions 
into a single tactile image. In addition, it made for more impressive demonstrations. 


4.2 Representation: Describing How Something Feels 


In designing the recognition program, l began by developing a simple language for 
describing the tactile properties of an object, 'the source of ideas tor this first-pass tactile 
description language was introspection. What properties do I notice when i feel an object?. After a 


few minutes of rolling the test objects under my (biological) fingci, 1 noticed three categories o* 
features that I was measuring. These are parameters 1 choose for representing the feel of an object. 
Here is a list of tire of the parameters- 


SHAPE- The general shape of die object. For the fastener micro-world there are only two 
possible shapes: ROUND and LONG. The shape may therefore be determined directly from 

the aspect ratio of the image. 


BUMPS- 'fhe locations of local pressure anomalies. These locations arc expressed in object 
relative coordinates, for example, in coordinates relative to the major and minor axes of the 
image. Bump may include both positive bumps that stick out, and negative bump (holes) 
that stick in. (In the implemented program, only the sign and position of a hump are 
considered significant for matching purposes. There is no intensity information.) 
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STABILITY- This property indicates how easy it is to roll the object in various directions, (in 


the program, two boolean values represent the stability of an object. These indicate whether 
or not the object rolls freely along each of the primary axes.) 


These three parameters proved sufficient to distinguish between any of the objects in 
the test set. Figure 9 shows which properties each of the objects exhibits. 


4.3 Implementation 


Given a representation of the objects in question, how do you write a program that 
measures the relevant features? This section describes a first pass that 1 have made at writing such 
a program. 

The routines described below have all been written and tested, but the complete sequence of 


sub-steps has not yet been strung together into a single program. I will begin with a description of 
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the hardware on which the program runs, because some of it was specifically designed for this 
application. 


4.3.1 The Hardware 


The control tnsks circ divided between n lisp Machine ond nvc Z80.rnicio processors. 
Most of the code is written in i.isp, and runs on the Lisp Machine. The micro-processors are each 
dedicated to a simple task with real time constraints. One scans the tactile i mage array, while 
others take specifications for joint motions, and execute them in real time. S he memories of the 
micro-processors are directly accessible by the Lisp Machine. 1 his shated memory piovidcs a 


con 


venient interface between the two levels of processing. The output of the tactile array, for 


example, is available as a Lisp array object. A lisp program may treat this array just as any other 

/ 

array: the memory sharing is invisible to the software. 

Another example of the use of shared arrays is in the control of joint motions. A lisp 
program computes the desired joint motions, or chooses them from a library of precomputed 
motions. Each motion is stored as a vector of joint angles or torques, representing the successive 
states of the joint over a period of time. These vectors are one dimensional arrays, shared between 
the two machines. The actual control of the motors is handled by the micro-processor, using the 
specifications in the shared arrays. Real time control would be difficult to realize on the Lisp 
Machine alone, because of virtual memory swapping, and multiple-process scheduling. 



Written in LISP, the top level of the recognition program 


like this- 


\ 

* 
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(defun recognize {) ;debug loop 

(do ((setq assumed-image (best-match))) ; hypothssize 

((test-as sumptions)))) ;test 

(defun test-assumptions () 

(do ((] (funcall real-image unknown-attributes) (cdr 1))) 

((null 1) t) 

(or (similarp (funcall real-image (car 1)) (funcall assumed-imago (car 1))) 
(return nil)))) ;try to verify each assumption 


This program is just a translation of the "Active Sensing" flowchart shown is Figure 10. 


It begins by assuming that the object is whatever best matches the information that it has so far. 


This is the "hypothesize" step. It then tests assumptions that it has made about the object, and 
unless they are all correct, repeats the process from the beginning. Testing the assumption may 


involve moving the finger to roll or probe the object, or may just involve making a computation 


from already collected data. 

Flow of control in this process depends on what information is needed. When a property 
is queried, if it is not already known, it is computed or measured. In the process of computing the 


value, the program may make queries about properties, which in turn may need to be computed. 


For example, if we ask an object's shape, and it is not known, then it must be computed from the 
object’s dimensions along the primary axes. If these axes arc not known, they must be computed 
from the image of die object. If no image has been read in, the finger must be pressed against the 
object. And so on. This is "call by need" control flow. It prevents information from being 


measured or computed unless it is actually needed in the recognition process. 


4.3.3 Crunching an linage 


After an image is read in, it must be processed to determine the object’s location, the 
primary axes, and the location of any bumps. The first step in processing the tactile image is the 


elimination of unwanted detail. This is accomplished by convolving the image with a simple pulse 
function. The image is contrast-enhanced to two bits per pixel by comparing it to fixed threshold 
values. If the offset pressure was chosen properly (sec below) the four possible pressure values 
correspond to background, depressions, primary figure and bumps. 
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Ncxt the aspect ratio and major and minor axes of object arc determined. Computation 

I 

of the aspect ratio begins by first locating die center of the activated area. For this purpose all 
non-zero pixels are taken to be part of the object. 'Hie center and the point farthest away from it 
determine the major axis of the object. The minor axis is taken to be perpendicular to this. Those 
axes provide an an object-relative coordinate system in which it is possible to specify, roughly, the 
location of bumps and depressions in the image. The bounding rectangle of the object is taken to 

be the smallest rectangle, with edges parallel to the axes, which contains the image. The aspect 

' \ ' ^ ■ 

ratio of the object is taken to be the aspect ratio of its bounding rectangle. 


4.3.4 Moving the Finger 


If the image read in is not satisfactory, it is possible to move tire finger and lead 
another-- we do not have to rely on first impressions. An important pait of the image analysis 
involves moving the finger so that an optimal image is sensed. The offset pressure, for example, 
may be adjusted in this manner. Optimally, most of the touched area activated the mid tango of 
the sensor, allowing bumps and depressions to be easily detected. This is accomplished by reading 
in an image, computing the median pressure of all points above tire noise threshold, and 
readjusting the finger pressure appropriately. This may be repeated scveial times until an 
acceptable offset pressure is achieved. 

The finger is also moved to measure the stability (resistance to roll) of an object, lo 
measure the object’s stability in a given direction, the object is pressed between die finger and the 
supporting surface by applying a fixed force on tire object normal to the plane of the surface. The 
finger is then moved laterally in the desired direction. (The supporting surface should have a high 


coci 


efficient of friction on the object to prevent sliding .) the stability of the object is indicated by 


the amount of force necessary to move the finger. 






■unri 
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4.3.5 The Matcher 




During the hypothesize step, the program must determine which of tltc objects it knows 
best matches the known data. For a small possibility set, such as the fasteners, it is not really 
important that this is done well. For a large set of possible objects the quality of the matcher may 
he a determining factor in the speed of recognition. When hypothesis is chosen by selecting the 
possibility that best matches the information given, usually the choice that has the largest number 
of features in common with the knov. n facts is the best choice. In a system with a large number of 
parameters other factors may also be taken into consideration. For one thing ;omc features may 
be more important than others, cither in general or for that particular possibility. Also, the 
features themselves may not exactly match-- a bump may be too large, a shape distorted. In cases 
such as this vve wish to give the possibility only partial credit for a feature match. 

The most obvious way to implement such a matcher would be a numerical scoring 
system with the weighting factors for feature importances and partial matches. I avoided such a 
solution for two reasons. First, there would have to be a degree of arbitrariness in assigning the 
numbers: Is a circle a 50% match to a hexagon? Is shape 2.5 times as important as texture, or only 
twice? it is unwise to trust the sums and products of numbers if the numbers themselves are 
chosen arbitrarily. The second objection is more of a philosophical one- converting a complex set 
of symbolic structures into a single number throws,away too much information, too quickly. Of 
course, this information must eventually be lost- the matcher must terminate by selecting a single 
item. But the pruning can be, and is, controlled in a more reasoned manner. 

The implemented matcher takes two possibilities at a time, and compares them on a 
feature by feature basis. If, for a particular feature, both items match tire image to about the same 
degree, the information is ignored. If one of tire items is clearly a better match, the feature is 
counted in favor of the appropriate item. This procedure is repealed for each feature and then the 
features themselves are compared in a similar manner. A feature counted toward one item will 


since! with a feature counted toward another, if they arc of approximate importance. 
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1 am assuming that in a large AI system the best match would be computed in parallel. 
Parallel marker propagation schemes, such as the one proposed by i-ahiman [2], would do such a 
task well. One important assumption, even for die parallel case, is that the binary comparison 
operator is transitive. Without this constraint it would be necessary to compare each possible pair 

of items, a task which grows as the square of the number of items. 

The transitivity of the predicate described above can be easily demonstrated, given the 
transitivity of individual feature comparisons. Assume there exist three items A,B, and C such 
that A > B and B > C. Let f(x,y) be the set of features counted in favor of x when compared with y. 

Since the individual feature comparisons arc transitive, f(A,C) = (f(A,B) U f(B,C)) and f(C,A) = 
(f(C.B) U f(B,A)). If > is the feature set comparison predicate (the second stage of the algorithm 

above), then A > B implies f(A,B) > f(B,A. Also, for any sets a,b,c and d such that a > b and c > 
d, it must be that (a U c) > (b U d), because features that cancel in the individual sets will also 

cancel in the union. The assumptions , A > B and B > C, imply f(A,B) > f(B,A) and f(B,C) > 
f(C,B), and by the union rule (f(A,B) U f(B,Q) > (f(C,B) U f(B,A)). This may be rewritten as 
fi'A.C) > f(C,A), which is the criterion for A > G. Therefore, the matching predicate is transitive. 

This matcher is really overkill for a possibility set of six objects with thice parameters 
each, but it may be necessary if the program is to be extended to a large range of objects. 


4.4 Limitations, What Needs to be Done Next 0 


One should not be too impressed by a program that distinguishes between six objects on 
the basis of three parameters. If only a single bit of information was derived from each parameter, 
it should be enough to recognize at least eight objects. In the future, tactile recognition piogiams 
will have more complex and more precise representations of tactile images. In this last section, I 
would like to mention three improvements that 1 believe arc just around the corner. 

The first is texture recognition. Tire resolution of the tactile array sensor, while high, is 
grossly insufficient for measuring textural differences between, say, paper and glass. Texture 




-33 


S 


ensing, require the detection of bulk effect of multiple surface features. It is most easily 


accomplished sliding something over the surface and noticing the pattern of vibrations, in much 
the same way that we slide a phonograph needle over a record. In fact, I have done some 
preliminary experiments using just that, a phonograph needle. Sensor of the future may use 


n 

v 


mbedded piczo-clcctric devices, or it may be possible to use the ACS directly as sort of a carbon 


microphone. However the information is derived, it must be processed into a useful 
characterization of the texture of the surface. One of the things we arc interested in is the intensity 
and periodicity of the signal. These features may be seen directly in the frequency domain. 
Texture processing may bear more similarity to the analysis of sounds titan to the analysis of 

visual images. 

Another difference between paper and glass is that glass feels cold. 1 his is not actually 
because that glass is lower in temperature, but because it is a better conductor of heat and so it is 
able to more quickly carry away die beat generated by the body. I have constructed a small 
thermal conductivity sensor that works on this principal. In the sensor, a resistive heating element 
is sandwiched between two temperature sensitive current sources. Any difference in the 
temperature of the two sensors is indicated by an easy to measure difference in the currents. The 
sensor is designed to be mounted on the finger in such a way that one temperature sensor may 
contact the device being tested. As the beat is drawn from the object into the object, a difference 
in temperatures will develop. The primary disadvantage of this first prototype is that it is large ( 0.1 
inches x 0.3 inches x 0.2 inches) resulting in a relatively high thermal mass. This limits both the 
response time and the minimum size of object which may be usefully tested. I believe the time is 
ripe for more work in tins area. 

The third area which shows immediate potential for further research is tire coordination 
of multiple tactile images into a global picture. 1 deliberately avoided this problem in my studies 
by choosing a small object that could be read in a single impression. Restricting the range possible 
of objects to this degree imposes limitations that may be unacceptable outside of the laboratory 


/ 
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/ironmcnt. I have done no work in this area, but f believe it to be an approachable, solvable 


problem. 


1 am enthusiastic about the future prospects of automated tactile sensing. The field is 
begging for more work. What has been described here- die sensor, the finger, the program- only 
scratches die surface of what is possible, I lie mechanical hand of thefuture will have a sense of 


touch. 
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